Background Readings for Collection Synthesis

Automatic collection building will be key to efficient and affordable digital libraries of the future. Since hand-curated collections will not scale with the Web, effective automated techniques are highly desired. This section deals with crawling the Web to extract topics, related documents, or collections. To do that, technologies such as feature extraction, page categorization, andWeb crawling are needed. This section covers these technologies. See also the section below on “Knowledge Extraction and Organization”, which also has to do with finding information on the Web. We are simultaneously applying these techniques in a collection building project for the National Science Digital Library[51, 49, 98, 109]. The issue of generating metadata to describe these collections is at the moment a to-be-done research topic.